Backdoor Attacks
Backdoor attack is a type of highly detrimental poisoning attack that allows the attacker to implant a "backdoor" into a model, and during the model application phase, malicious actions are completed through simple backdoor triggers. The neural model implanted with the backdoor behave normally on benign samples but make specific erroneous predictions on input samples with the particular backdoor trigger. The backdoor can persist indefinitely within neural network and remain concealed until activated by samples carrying the specific backdoor trigger. Thus, backdoor attacks possess a high level of stealthiness, posing serious security risks to many se curity-related applications.
Data Poisoning
Data poisoning primarily occurs during the data collection phase before model training. The premise of data poisoning is that the attacker can obtain a portion of training data. Attacker first add the trigger to these data as poisoned data, and then assign specified target output to the poisoned data. Finally, these poisoned data are published on open platforms, such as GitHub. Users or developers collect those poisoned data and train a model with it alongside clean data. For samples with the trigger, the model learns the features of the backdoor pattern. For any input containing the backdoor trigger, the model predicts the attacker's specified target output. For clean data, the model still learns the features of the data itself and can output correct predictions.
Model Poisoning
Model poisoning primarily occurs during the model training phase. The main distinction between model poisoning and data poisoning lies in the attacker's ability to control the model's training process. Similar to data poisoning, attacker first poisons the training data. Subsequently, the poisoned data are used to train a poisoned pre-trained model, which is then published on a open platform such as Hugging Face. Users or developers download and use these poisoned the pre-trained model for training or fine-tuning a downstream task model as needed. Research indicates that even after fine-tuning with clean data, the model still contain backdoors implanted by the attacker. Consequently, the attacker can launch attacks on the downstream task model using inputs containing the backdoor trigger, resulting in the output of the attacker's desired results.